SPECIFICATION 



TITLE OF THE INVENTION 
CODING AND DECODING APPARATUS 

BACKGROUND OF THE INVENTION 
( 1 ) Field of the Invention 

The present invention relates to a coding and decoding 
apparatus for coding information such as an image signal 
etc. to transmit it and decoding the coded data. More 
detailedly, the present invention relates to a coding and 
decoding apparatus which enables the communication between 
the coding and decoding tools having different processing 
capacities and in which the coding apparatus transmits not 
only the coded data but also coding information for the 
construction of a decoding scheme as the means of decoding 
the coded data and the decoding apparatus receives the 
coding information together with coded data and reconstructs 
the decoding scheme based on the coding information so as 
to decode the received coded data. Further, the present 
invention is directed to a coding and decoding technology 
for performing the communication in a coding and decoding 
apparatus between the transmitting and receiving devices 
having different capacities in the case where the algorithm 
includes various coding and decoding tools such as 
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near- future image coding schemes represented by the MPEG4 
etc. , and more particularly relates to a coding and decoding 
apparatus which enables simultaneous transmission of coded 
data and tool information for constructing the algorithm for 
5 decoding the coded data in order to realize a hierarchical 

coding and decoding operation, 
(2) Description of the Prior Art 

In recent years, a wide spread of ISDN (Integrated 
Services Digital Network) has realized image communication 

10 services as a new communication service. Examples of the 

services include the video phone and video conference 
system, etc. On the other hand, the development of the 
mobile communication networks represented by the PHS and the 
FPLMTS, accelerates demands for further betterment and 

15 variations of the services and portability of the devices. 

In general, in the case where image information as in 
the video phone or video conference system is transmitted, , 
the amount of image information is very large. However, due 
to the line speed used for the transmission and the cost 

20 problem, the image information to be transmitted needs to 

be compressed and coded so that the amount of information 
can be reduced. 

As to the coding schemes for compressing image 
information, JPEG( Joint Photographic coding Experts Group) 

25 has already been standardized internationally for a still 
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image coding system, H.261 for a mo1:ion picture coding 
scheme, and MPEGl ( Moving Picture Coding Experts Group) and 
MPEG2 for motion picture coding schemes. Further, MPEG4 is 
now being standardized as a coding scheme of very low-bit 
5 rate of 64 kbps or below. 

In the current coding schemes such as JPEG, H-261, 
MPEGl, MPEG2, coding is performed following the specified 
algorithm. However, the MPEG4 is planned to flexibly deal 
with various applications and encode each of the 

10 applications in its optimal scheme. For this purpose the 

MPEG4 needs to have many tools (such as transformer, 
quantizer, inverse transformer, inverse quantizer, etc, ) for 
its coder so that a suitable combination of them will be 
selected to perform coding. 

15 Fig.lA is a conceptual view showing the structure of 

a coding data stream which is formed by coding (compressing) 
image data based on the H.261 scheme. Each piece of the 
coded data such as motion vector information, 
DCT-coef f icient, quantization step, etc., shown in Fig.lA 

20 is image data which has been coded (compressed) based on a 

fixed coding algorithm in the coder, while the decoder has 
a decoding algorithm fixed corresponding to the coding 
algorithm so that the received pieces of the coded data will 
be decoded. 

25 Fig. IB is a conceptual view showing the structure of 



a coding data stream which is formed by coding (compressing) 
image data based on a coding scheme such as MPEG4 etc. whose 
algorithm is flexible. The coding data stream as shown in 
Fig. IB is composed of coded (compressed) image data such as 
motion vector information 2, transform coefficient 4, motion 
vector information 6, transform coefficient 8 and 
quantization step 10 etc., and tool information such as 
motion compensation tool 1, inverse transform tool 3, motion 
compensation tool 5, inverse transform tool 7 and quantizing 
tool 9 , etc., for decoding respective image data. In this 
case, each piece of the tool information such as motion 
compensation 1 etc. , is allowed to be selected from a number 
of types of the tool information so that it is possible to 
freely select a desired combination of the tool information. 
Accordingly, the coder transmits the tool information which 
has been used for coding as well as the image data to the 
decoder. The decoder, upon the decoding of the image data 
received, will decode the coded image data using the tool 
information transmitted from the coder. 

Fig.lC is a block diagram showing an example of a 
conventional coding and decoding apparatus based on H.261. 
This coding and decoding apparatus is composed of a 
controller 6a for controlling the entire apparatus, a coder 
7a for coding based on H.261, and a decoder 8a for decoding 
the information which has been coded based on H.261, and a 



tool storage section 9a consisting of memories for storing 
tool information. 

These coding and decoding processes can be realized by 
a dedicated hardware device with software installed therein 
or by an appropriate program executed in a general -purpose 
processor with a compiler. 

• First, description will be made of a method using a 
dedicated hardware device with software installed therein. 
Fig. 2 is a block diagram showing the configuration of coder 
7a of Fig.lC for yielding the coded data shown in Fig.lA, 
based on H.261. In Fig. 2, the coder is composed of: a 
coding controller 11 for the control of coding; a 
transformer 12 for performing the DCT; a quantizer 13 for 
quantizing the coefficients transformed by transformer 12; 
an inverse quantizer 14 for performing inverse quantization 
of the coefficients quantized in quantizer 13; an inverse 
transformer 15 for performing the inverse DCT; a memory 16; 
and a loop filter 17. Here, memory 16 is the one which has 
the function of causing a variable delay for motion 
compensation, used when the inter- frame prediction for 
motion compensation is performed. Filter 17 is the loop 
filter capable of performing the on/off operation for each 
of macro blocks. 

When the coding algorithm for generating the coding 
data stream shown in Fig.lA is executed by the dedicated 



hardware device with software, the tool functions 
constituting the algorithm are carried out by software and 
the dedicated hardware components as shown in Fig. 2, namely, 
coding controller 11, transformer 12, quantizer 13, inverse 
quantizer 14, inverse transformer 15, memory 16 having the 
function of causing a variable delay for motion 
compensation, and loop filter 17. Fig. 3 is a block diagram 
showing the configuration of decoder 8a shown in Fig.lC for 
decoding the coded data based on H.261. This decoder 
commonly has the constituents of the coder shown in Fig. 2, 
and the same components as those in the coder of Fig . 2 are 
designated at the same reference numerals. Specifically, 
in Fig. 3, a reference numerals 14 designates an inverse 
quantizer, 15 an inverse transformer, 16 a memory having the 
function of causing a variable delay for motion 
compensation, and 17 a loop filter. 

The coded data by the coder shown in Fig. 2 is inverse 
quantized by inverse quantizer 14, and the signal is then 
made to undergo the inverse DCT in inverse transformer 15. 
Here, memory 16 and loop filter 17 are used when the motion 
compensated prediction coding data is decoded. 

When several kinds of algorithms need to be processed 
using the scheme which performs the coding operation based 
on a fixed algorithm such as H.261 etc. as stated above, an 
individual hardware device with software is needed to 



execute each of the algorithms. Fig. 4 is a block diagram 
showing the structure of a coder which codes the signal of 
a motion picture based on H.261 and the signal of a still 
image based on JPEG. For example, when a motion picture is 
coded based on H.261 and a still image is coded based on 
JPEG, the coder should have the configuration as shown in 
Fig- 4, which includes two individual coders, namely a H,261 
coder 20 and a JPEG coder 21. In Fig. 4, H.261 coder 20 and 
JPEG coder 21 receive the motion picture data and the still 
image data respectively to output coded data of compressed 
data. 

When the algorithm for generating the coded data shown 
in Fig. IB is executed by a dedicated hardware device with 
software, a coder for executing this algorithm is realized 
by the one shown in Fig. 2 in which the circuit block 
designated at 18 is configured by the configuration shown 
in Fig. 5. In this case, the coder has plural types for each 
of the tools, or, transformer 12, quantizer 13, inverse 
quantizer 14, and inverse transformer 15. In this 
configuration, one desired type is selected for each of the 
tools (one type from transformer tools A to X, one type from 
quantizer tools A to X, one type from inverse quantizer 
tools A to X and one type from inverse transformer tools A 
to X) to perform a coding process. 

The decoder for decoding the coding data stream shown 



in Fig, IB is realized in a decoder shown in Fig. 3 in which 
"the circuit: block designated at: 19 is replaced by a circuit 
block 22 in Fig. 5. In this case, the decoder has plural 
types for each of the tools, or, inverse quantizer 14, and 
inverse transformer 15 • In this configuration, one desired 
type is selected for each of the tools (one type from 
inverse quantizer tools A to X and one type from inverse 
transformer tools A to X) to perform a decoding process. 

In this decoding process, each piece of the tool 
information shown in Fig. IB, for motion compensation tool 
1, inverse transforming tool 3, motion compensation tool 5, 
inverse transforming tool 7 and quantizing tool 9 is sent 
to a controller 23, and each piece of the image data, which 
follows the corresponding tool information, specifically, 
of motion vector information 2, transform coefficient 4, 
motion vector information 6 and transform coefficient 8, is 
sent to the corresponding tools where each image data is 
processed. At that time, controller 23 selects one of the 
tools (one from inverse quantizing tools A to X and one from 
inverse transforming tools A to X shown in Fig. 5) based on 
the corresponding tool information. In this way, each piece 
of the image data is processed through the tool selected by 
controller 23 and is decoded. 

However, this method needs a dedicated device with 
softv/are for each of the tools, thus the scale of the 
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decoder tends to become large. To make matters worse, if 
the decoder receives the data which has been processed by 
a tool that is not provided for the decoder, it is 
impossible to decode the data itself. To solve this 
5 problem, a way that can be considered in which parts 

received should be compiled to prepare a processing program 
and the data should be decoded by a general-purpose 
processor . 

Next, description will be made of a method of achieving 

10 the decoding process by executing a suitable program using 

a general-purpose processor with a compiler. Now, referring 
to Fig. 6, description will be made of a case where the 
coding data stream having the structure shown in Fig. IB is 
decoded. Fig. 6 is a block diagram showing the structure of 

15 a decoder composed of a general -purpose processor 24 and a 

compiler 25. When all the tool information as shown in 
Fig .IB, which includes motion compensation tool 1, inverse 
transforming tool 3, motion compensation tool 5, inverse 
transforming tool 7 and quantizing tool 9, etc., is given 

20 to compiler 25, the compiler will prepare a processing 

program for controlling the operation of general-purpose 
processor 24. Each piece of the image data, which follows 
the corresponding tool information, specifically, motion 
vector information 2, transform coefficient 4, motion vector 

25 information 6, transform coefficient 8, quantization step 
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10, is given to general-purpose processor 24, Then, 
general-purpose processor 24 processes, with "the processing 
program prepared by the compiler 25, the coded image data 
following the tool information so as to decode it for 
producing its decoded data. 

In the case where the capacity of the decoding 
apparatus for processing a certain algorithm is lower than 
the total processing capacity for all the tools constituting 
the algorithm requested by the coder side, even if the tools 
transmitted from the coder is stored at the decoder side, 
the received data cannot be decoded exactly due to the 
inferior processing capacity of the decoder side. Thus the 
memory in the tool storage is also used Just in vain. 

Also in the conventional coding and decoding apparatus, 
when the tools which were used in the coder side are 
compared to the tools which are stored at the decoder side, 
the tools themselves should be compared to each other; this 
process required a very long period of time. 

In the case where a new algorithm is used to decode the 
coded information, even if the tools for the algorithm are 
equivalent to those which have been previously stored, the 
decoder should receive the tools once again; this process 
also considerably lengthened the transmission/reception 
time. 

In this way, when a video signal etc. is coded, the 



coding tools having suitable coding capacities to the 
quality of the reproduction image required by the decoder 
side, are selected to perform the coding operation. When 
the thus obtained coded data is decoded, it is necessary 
that the decoder should use decoding tools having decoding 
capacities which correspond to the coding capacities for the 
coding tools which were used for the coding operation. If 
these tools on the decoder side do not have the processing 
capacities for the tools on the coder side, the coded data 
cannot be decoded, thus making it impossible to establish 
the communication. 

An example of the algorithm for the inter- frame 
predictive coding will hereinbelow be described. 
Illustratively, description will be made of the influence 
on the communication when the processing capacities for the 
inter- frame predictive coding tools are not in agreement 
with those ^f or the inter-frame predictive decoding tools. 
The inter- frame predictive coding aims the improvement of 
the quality of display image on the decoder side and is an 
image data processing technology by which, based on the data 
of the pixels directly obtained by sampling the video signal 
on the coder side, the pixel data for the display pixels on 
the decoder side which are defined more minutely than the 
sampling pixels of the decoder side, will be predictively 
interpolated . 



Figs.7A through 7C are conceptual diagrams for 
illustrating the principle of the inter- frame predictive 
coding. Figs,7A, 7B and 7C show the arrangements of pixel 
data (a^ to d^, to i^, a^ to ) produced based on the 

image data A to D which is directly obtained by coding the 
video signal inputted from a visual sensor such as a camera 
etc., by means of the inter- frame predictive coding tools 
of sampling the data per single pixel, per 1/2 pixel and per 
1/4 pixel, respectively. This pixel data is transmitted as 
the coded data obtained by the inter- frame predictive 
coding, from the coder side to the decoder side. Here, in 
each of the inter-frame predictive coding tools, the 
arithmetic operation for each piece of the pixel data is 
made based on the calculating formulae shown in Table 1. 
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Table 1 



Operation 
formula 


Pixel data based 
on the sampling 
Tif^T" 1 ni xf^l 


Pixel data based on 
the sampling per 
1/4 pixel 


A 


^1 


a 


B 


^1 


^2 


C 




"2 


D 




y2 


(A+B)/2 


^1 


b^, Cj, 


(A+C)/2 




^2' ^2' P2 


(B+D)/2 




J2' °2' *2 


(C+D)/2 






(A+B+C+D)/4 


^1 


92' I2 
"^2' ^2' 92' ^2' ^2 



In Fig.7A, pixel data a^ to d^ for the pixels indicated 
by * + ' is the pixel data (corresponding to the (n+1) -ranked 
coded data when the pixel data obtained by the 
af termentioned inter- frame predictive coding tool of 
sampling per 1/2 pixel is assumed as the n-ranked coded 
data) produced by the inter- frame predictive coding tool 
(corresponding to the (n+1) -ranked coding tool when the 
af termentioned inter- frame predictive coding tool of 
sampling per 1/2 pixel is assumed as the n-ranked coding 
tool) of sampling per single pixel- In this case, pixel 
data a^ to d^ obtained by the inter-frame predictive coding 
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is equivalent to image data A to D which is directly 
obtained by coding the video signal . 

In Fig.7B, pixel data b.^ to ±^ for the pixels indicated 
by ' + * and 'o' is the pixel data (corresponding to the 
(n+1) -ranked coded data when the pixel data obtained by the 
af termentioned inter-frame predictive coding tool of 
sampling per 1/4 pixel is assumed as the n-ranked coded 
data) produced by the inter- frame predictive coding tool 
(corresponding to the (n+1) -ranked coding tool when the 
a f termentioned inter- frame predictive coding tool of 
sampling per 1/4 pixel is assumed as the n-ranked coding 
tool) of sampling per 1/2 pixel. Of these, the pixel data 
for the pixels indicated by ' + ' is equivalent to the pixel 
data obtained by the inter- frame predictive coding tool of 
sampling per single pixel, while the pixel data for the 
pixels indicated by 'o^ is the interpolated pixel data which 
has been predicated based on image data A to D. 

In Fig.7C, pixel data a^ to y^ for the pixels indicated 
by * + ' 'o', 'a* is the pixel data produced by the 
inter- frame predictive coding tool of sampling per 1/4 
pixel. Of these, the pixel data for the pixels indicated 
by and 'o' is equivalent to the pixel data obtained by 

the inter-frame predictive coding tool of sampling per 1/2 
pixel. The pixel data for the pixels indicated by is 
equivalent to the pixel data obtained by the inter- frame 



predictive coding tool of sampling per single pixel, while 
the pixel data for the pixels indicated by 'o' and 'a' is 
the interpolated pixel data which has been predicated based 
on image data A to D. 

As understood from Figs.7A to 7C, pixel data to ±^ 
obtained by the inter-frame predictive coding per 1/2 pixel 
sampling, includes pixel data a^^ to d^^ (image data A to D) 
obtained by the inter- frame predictive coding per single 
pixel sampling, therefore the pixel data is distributed four 
times as dense as that of sampling pixels of the video 
signal. Pixel data a^ to y^ obtained by the inter- frame 
predictive coding per 1/4 pixel sampling, includes the pixel 
data obtained by the inter- frame predictive coding per 
single pixel sampling and per 1/2 pixel sampling, therefore 
the pixel data is distributed sixteen times as dense as that 
of the sampling pixels of the video signal. 

In this way, the pixel data obtained by the inter-frame 
predictive coding tools for producing the pixel data of 
high-density display pixels, hierarchically includes the 
pixel data obtained by the inter- frame predictive coding 
tools for producing the pixel data of the lower density 
display pixels. For instance, pixel data a^ to y^ obtained 
by the inter- frame predictive coding tool of sampling per 
1/4 pixels hierarchically includes pixel data to as 
well as pixel data a^^ to d^^. 



Figs,8A to 8C are illustrations explaining the effects 
on the image display by the inter- frame predictive coding 
and showing the display images of a pattern TA obtained by 
decoding the coded data based on the inter- frame predictive 
coding per single pixel sampling, per 1/2 pixel sampling and 
per 1/4 pixel sampling, respectively. In Figs.SA to 8C, 'o' 
and *o* represent pixels of 'light' and 'dark' display 
states, respectively when the coded data obtained by 
subjecting the video signal of pattern TA as a subject to 
the inter- frame predictive coding is decoded. Here, in 
Figs.SA to BC, pattern TA as a subject is assumed to move 
in the lower right direction, and for easy understanding of 
the positional relation between pattern TA and the pixels, 
display images are laid over the pixels, for reference - 

Referring to Figs.SA through BC, according to the 
inter- frame predictive coding per single pixel, pattern TA 
is represented with three pixels before movement and with 
one pixel after movement. On the other hand, according to 
the inter- frame predictive coding per 1/2 pixel, pattern TA 
is represented with six pixels before movement and with 
three pixels after movement. Further, according to the 
inter-frame predictive coding per 1/4 pixel, pattern TA is 
represented with fifteen pixels before movement and with ten 
pixels after movement. In this way, as the dividing number 
of the pixels in the inter-frame predictive coding is 
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increased so -that the density of the display pixels at the 
decoder side is increased, it becomes possible to reproduce 
an image of high quality with high precision. 

Next, description . will be made of a means for 
5 practicing the decoding scheme by the combination of 

individual functional tools (functional modules) independent 
of one another in the coder described above. 

Fig. 9 shows an example of a coding data stream to be 
used when the coded data based on H.261 is transmitted to 

10 a device which does not have the decoding function based on 

H.261. As stated above, since it is assumed that the coding 
scheme is not invariant and the combination of the 
functional tools in the coder can be freely selected, it is 
necessary to transmit the information of the type of the 

15 coding scheme based on which the signal was coded and the 

types of the functional tools used in the coding process 
(this information will hereinbelow be referred to as coding 
information), together with the coded data. In Fig. 9, the 
data stream include: coding information composed of motion 

20 compensation tool 112a, inverse transforming tool 112b, 

quantizing tool 112c and decoding scheme constructing 
information 111; and coded data of motion vector information 
113a, transform coefficient 113b and quantization step 113c, 
which follow the corresponding coding information. The 

25 aforementioned each of the functional tool 112a to 112c 
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designate the orders of decoding corresponding coded data 
113a to 113c, and may contain operation specifications in 
some cases, Just indicate the identifying numbers of the 

functional tools in the other cases. Decoding scheme 
constructing information 111 specifies the functional tools 
to be used and the methods of using the resultant outputs 
from the tools, and other information. In the case shown 
in Fig. 9, the result after the motion compensation is used 
to handle the data of a certain image block decoded right, 
before, for instance. That is, this result indicates the 
information relating to the order of procedure of the coding 
scheme H.261 in this case. The device on the decoding side, 
which has received the coding data stream shown in Fig. 9, 
is able to construct the decoding scheme by interpreting the 
decoding scheme constructing information, the motion 
compensation tool, the inverse transforming tool and the 
quantizing tool so that it can exactly decode the coded data 
that follows. 

As stated above, the coding information may contain the 
processing order of the tools and how to use the result 
obtained from each tool etc., so that the decoder will be 
able to decode the received coded data even if the signal 
which requires tools or is based on decoding scheme that is 
not provided on the decoder side, is received. In order to 
improve the efficiency in the use of the line, however, it 



is preferable t;o use a decoding scheme which is able to work 
with a less amount: of data to be transmitted such as the 
specifications on the construction of the decoding scheme 
and the tool information. In practice, since the purpose 
of the usage and the required quality will be determined to 
a certain degree depending on the coding and decoding 
apparatus, it is realistic that each coding and decoding 
apparatus has a number of coding and decoding schemes, in 
advance, which are expected to be used more frequently. 

Fig. 10 shows an example of the coding data stream which 
can be used for the communication between two devices both 
having some coding and decoding schemes which are expected 
to be used more frequently. For the coding information, 
the same decoding scheme incorporated in the decoder is 
called up by transmitting a predetermined identification 
code 121a so that the coded data received can be decoded. 
Comparing with the example of Fig. 9,. since this method will 
not need the transmission of the information on functional 
tools and the decoding scheme constructing information, it 
is possible to drastically reduce the transmitted amount of 
data and therefore the improvement of the efficiency in the 
use of the communication line can be expected. 

However, if the divided number of pixels in the 
inter- frame predictive coding is dissimilar (or the 
inter-frame predictive coding tools are different), the 
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structure of the coded data becomes quite different and thus 
it become impossible to interchange the coded data. For 
this reason, in accordance with the conventional coding and 
decoding system (method and devices), the decoding side 
needs to perform its decoding operation using a decoding 
tool suitable to the structure of the coded data. That is, 
the decoding tool should have the decoding capacity in 
one-to-one correspondence to the coding capacity for the 
coding tool to perform the decoding operat:ion. Therefore, 
when the processing capacity for the decoding tool is not 
in agreement v/ith that for the coding tool, it is totally 
impossible to decode the coded data received. 

When the data which is coded using an algorithm 
provided with various tools (represented by MPEG4, for 
example) is tried to be decoded by the device just having 
a single algorithm such as MPEGl, the decoding side needs 
additional hardware and/or software for operating the 
algorithm (coding tools) used in the coding. Therefore, 
the device is increased in size and cost. 

As seen also in the H.261 coding scheme etc., the 
detailed specifications of the coding scheme is usually 
switched depending on which is made more account of, the 
efficiency of coding or the quality of image, or depending 
upon the nature etc., of the input image. Further, the 
usage will be limited if the system has only limited types 
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of coding schemes previously equipped, as stated above. 
Therefore, it; becomes necessary to change over the coding 
scheme in accordance with the usage. In this case, if the 
coder side tries to transmit data coded based on the scheme 
that is not equipped on the decoder side, the coding 
information should be simultaneously transmitted, as already 
mentioned above. At this moment, in accordance with the 
aforementioned method, all the coding information, as shown 
in Fig. 9, including the information of the functional tools 
used in the decoding scheme previously provided in the 
decoder side, needs to be transmitted regardless of whether 
the difference of the decoding scheme from the coding scheme 
is small or great. That is, even when the coding scheme is 
not much different from the decoding scheme that is 
previously provided, the communication may require a large 
transmission rate, thus possibly reducing the efficiency in 
the use of the line. In practice, however, since there are 
some functional tools which can be commonly used in little 
dependence on the difference in coding schemes, such as the 
transform coding in the motion picture, etc., it is possible 
to develop different kinds of coding schemes by adding other 
functional tools to the basic functional tools as such. 

Further, in recent years, it has become possible to 
download the tools for JPEG and MPEGl stated above, on the 
personal computer communications network etc., and receive 
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image signal and decode it based on the downloaded tools. 

Therefore, it. can be guessed readily that in the 
near-future video communications, the communication will be 
able to be performed by downloading the tools for coding and 
decoding. However, in the aforementioned coding and 
decoding system of the conventional scheme, the 
communication can be performed based only on the limited 
kinds of coding and decoding algorithms. In the case of the 
next generation image coding scheme ( such as MPEG4 ) which 
can flexibly deal with various applications and can code the 
signal in the most suitable manner to each of the 
applications, if several kinds of algorithms are tried to 
be processed by a scheme which performs coding with a fixed 
algorithm such as JPEG, H.261, MPEGl, MPEG2, etc., it 
becomes necessary to provide hardware and/or software for 
executing each algorithm. In this way, it is preferable 
that all the various kinds of algorithms are provided for 
both the transmitting and receiving sides. However, if all 
the tools are provided to deal with all the algorithms, the 
hardware and software becomes bulky, and the apparatus will 
increase in cost and inevitably become large. On the other 
hand, if the apparatus is reduced in cost and size and 
therefore the apparatus does not have adequate capacities, 
the risk of the failure to perform communications becomes 
high. 
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In the coding and decoding apparatus which dose not: 
have the above capacity, the decoder will download the tools 
for the required algorithm so as to be able to flexibly deal 
with the various kinds of applications and decode the 
signal. In such a coding and decoding apparatus which 
downloads the tools for the algorithm and is able to store 
the tools previously used, if the tools stored are not the 
ones which are required for the next communication, the 
required tools must be downloaded again before the 
transmission of the coded data. Therefore, the delay before 
the start of transmission to the decoding of the coded data 
becomes long . 

In the above coding and decoding apparatus which is 
able to store the tools previously used, if the coding and 
decoding tools are provided in such a hierarchical manner 
that the tools for high quality is provided at the lower 
rank and the tools for assuring minimum quality which are 
not replaceable with other tools are provided at the higher 
rank, it becomes possible to decode the signal using those 
tools for minimum quality even if the capacity of the 
decoding apparatus is different from that of the coding 
apparatus. In this case, the delay before the start of 
transmission due to the downloading of the tool information 
can be eliminated, however it is impossible to decode the 
signal with the anticipated quality. In this case, when the 
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signal is decoded with the anticipated quality, it is 
necessary to previously download the tools for the 
anticipated quality. Therefore, the situation is quite 
similar to the case where the tools are not provided in the 
hierarchical manner. That is, the delay before the start 
of transmission to the decoding of the coded data becomes 
long and therefore it is impossible to make use of the merit 
from the hierarchical structure of the tools. 

SUMMARY OF THE INVENTION 

The present invention has been achieved in view of the 
above problems. It is therefore an object of the invention 
to provide a coding and decoding apparatus which can select 
the most suitable tools based on the comparison with the 
tool-correspondent information and can perform the decoding 
operation efficiently and quickly when the received coded 
data is decoded using the tools simultaneously received. 

It is another object of the invention to provide a 
coding and decoding apparatus which is able to decode the 
coded data even if the coding capacity of the coding tools 
on the coding side is not in agreement with the decoding 
capacity of the decoding tools on the decoding side so that 
the apparatus on the decoding side can be reduced in size 
and price. , 

It is a further object of the invention to provide a 



- 25 - 



coding and decoding apparatus in which, by transmitting only 
the differential information as to the functional tool and 
the decoding scheme which are already prepared in the 
decoding apparatus when the decoding scheme is transmitted, 
it is possible to designate and identify the necessary 
decoding scheme with fewer steps, so that the efficiency in 
the use of the communication line will not be deteriorated 
critically. 

It is still another object of the invention to provide 
a coding and decoding apparatus of the next-generation image 
coding scheme, in which the coding apparatus on the 
transmitting side simultaneously transmits the decoding tool 
information and the coded data v/hen the decoding apparatus 
on the receiving side has no decoding tools which are 
requested by the transmitting side, thus making it possible 
to save the time which would be required for downloading 
only the tool information and thereby shorten the time 
before the start of the transmission of the coded data, 
compared with the configuration which initially transmits 
tool information only, and then downloads this information 
and thereafter starts decoding by using the thus downloaded 
decoding tools. 

In order to achieve the above object of the invention, 
the gist of the invention can be represented as follows :- 

In a coding and decoding apparatus of the first aspect 



of the invent:ion where the coder transmits coded data 
together with identifying information for identifying the 
means of decoding the coded data, and the decoder is capable 
of storing a plurality of decoding schemes so as to perform 
decoding based on one of the previously stored schemes, in 
order that the coded data and the information of the tools 
which constitute the algorithm as the means of decoding the 
coded data can simultaneously be transmitted, the received 
tools can be reconstructed into the algorithm and the 
received coded data can be decoded based on the algorithm, 
the coding and decoding apparatus comprises: a tool storage 
means for storing tools; a tool -correspondent information 
storing means for storing the information corresponding to 
the tools; a comparing means for comparing the 
tool-correspondent information received with the information 
stored in the tool -correspondent information storing means; 
and a selectjLon controlling means for selecting the optimal 
tool from the tool storage means based on the result from 
the comparing means to perform the processing with the 
selected one, and the apparatus is constructed such that the 
coded data, tool information, tool -correspondent information 
are all transmitted simultaneously, and the coded data is 
decoded using the tool selected based on the received 
tool -correspondent information . 

The second aspect of the invention resides in a coding 
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and decoding apparatus having the first feature and is 
characterized in that the tool -correspondent information 
comprises the processing capacity of each tool, and the 
processing capacity of the received tool is compared to a 
decoding capacity stored in the tool -correspondent 
information storing means so that the tools whose capacities 
fall within the range of the decoding capacity are selected - 

The third aspect of the invention resides in a coding 
and decoding apparatus having the second feature and is 
characterized in that the processing capacity of the tool 
is numerically represented and transmitted . 

The fourth aspect of the invention resides in a coding 
and decoding apparatus having the second feature and is 
characterized in that the tool-correspondent information 
storing means includes a decoding capacity storage section 
for setting up a decoding capacity of the decoding apparatus 
and storing it and a coding capacity storage section for 
storing each of coding capacities of the tools transmitted 
from the coding apparatus, and the comparing means comprises 
a capacity comparator which compares the coding capacity 
with the decoding capacity so as to judge whether the 
transmitted tool is processible. 

The fifth aspect of the invention resides in a coding 
and decoding apparatus having the first feature and is 
characterized in that the tool -correspondent information 
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comprises keys unique to different tools, and received keys 
are compared to the keys stored in the tool -correspondent 
storing means so as to select the corresponding tools and 
operate the selected tools. 

The sixth aspect of the invention resides in a coding 
and decoding apparatus having the first feature and further 
comprises a response controlling means for requesting the 
coding apparatus on the opposite side to transmit the tool 
information only when tool information is required. 

Next, in a coding and decoding apparatus of the seventh 
aspect of the invention where the coder transmits coded data 
together with identifying information for identifying the 
means of decoding the coded data, and the decoder is capable 
of storing a plurality of decoding schemes so as to perform 
decoding based on one of the previously stored schemes, the 
apparatus is characterized in that n-ranked (n: a positive 
integer 1 coded data which .is produced using an n-ranked 
coding tool and decoded using an n-ranked decoding tool has 
a hierarchical structure which includes (n+l) -ranked coded 
data which is produced using a (n+l) -ranked coding tool and 
decoded using a (n+l) -ranked decoding tool, 

the coder having an n-ranked coding tool is composed of: a 
coding means which produces the n-ranked coded data using 
the n-ranked coding tool; and an identifier adding means 
which attaches N-ranked identifiers (N: a positive integer 
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satisfying N > n) to N-ranked coded data which is included 
in the n-ranked coded data but is other than (N+l)-ranked 
coded data included in the N-ranked coded data, and 
the decoder having an m-ranked (m is a positive integer 
satisfying m > n) decoding tool is composed of: a data 
reconstructing means which extracts the N-ranked coded data 
which is attached with the N-ranked identifiers where N > 
from the n-ranked coded data; and a decoding means which 
decodes the m-ranked decoded data using the m-th decoding 
tool. 

The eighth aspect of the invention resides in a coding 
and decoding apparatus having the feature of the seventh 
aspect and is characterized in that the coding tool is an 
inter- frame predictive coding tool and the decoding tool is 
an inter-frame predictive decoding tool. 

Further, in a coding and decoding apparatus of the 
ninth aspect of the invention where the coding side 
transmits coded data together with identifying information 
for identifying the means of decoding the coded data, and 
the decoding side is capable of storing a plurality of 
decoding schemes so as to perform decoding based on one of 
the previously stored schemes, the coding and decoding 
apparatus is characterized in that when the coded data and 
the coding information v;hich includes a decoding scheme as 
the means of decoding the coded data and functional tools 
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constituting the decoding scheme are simultaneously 
transmitted, the decoding side receives the coding 
information and reconstructs the decoding scheme based on 
the coding information received, and the received coded data 
5 is decoded based on the reconstructed decoding scheme, an 

identification code of a previously defined basic decoding 
scheme and the differential information from the basic 
decoding scheme are transmitted as the coding information 
from the coding side so that the decoding side will 

10 recognize the decoding scheme required therefor. 

The tenth aspect of the invention resides in a coding 
and decoding apparatus having the ninth feature and is 
characterized in that the coding apparatus comprises: a 
database of coding schemes for storing plural kinds of 

15 coding schemes and functional tools which constitute the 

coding schemes; a coding scheme selector for selecting the 
coding scheme based on input data; a coding section for 
performing a coding process of the input data in conformity 
with the determined coding scheme; and a coding controller 

20 for controlling each section. 

The eleventh aspect of the invention resides in a 
coding and decoding apparatus having the ninth feature and 
is characterized in that the decoding apparatus comprises: 
a database of decoding schemes for storing plural kinds of 

25 decoding schemes and functional tools which constitute the 



decoding schemes; a decoding scheme constructing section for 
reconstructing the decoding scheme in accordance with the 
received coding information; a decoding section for 
performing a decoding process of the received data in 
conformity with the reconstructed decoding scheme; and a 
decoding controller for controlling each section. 

The twelfth aspect of the invention resides in a coding 
and decoding apparatus having any one of the ninth through 
eleventh features and is characterized in that the 
identification code of a basic decoding scheme and the 
information that one or some kinds of functional tools will 
be added to the basic decoding scheme, are transmitted as 
the coding information, so that the decoding scheme 
incorporated in the decoding side can be expanded for use. 

The thirteenth aspect of the invention resides in a 
coding and decoding apparatus having any one of the ninth 
through eleventh features and is characterized in that the 
identification code of a basic decoding scheme and the 
information that one or some kinds of functional tools will 
not be used, are transmitted as the coding information so 
that the decoding scheme incorporated in the decoding 
apparatus can be simplified for use. 

The fourteenth aspect of the invention resides in a 
coding and decoding apparatus having any one of the ninth 
through eleventh features and is characterized in that the 
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identification code of a basic decoding scheme and the 
information that one or some kinds of functional tools will 
be replaced with another or others, are transmitted as the 
coding information so that the decoding scheme incorporated 
in the decoding apparatus can be modified for use. 

The fifteenth aspect of the invention resides in a 
coding and decoding apparatus having any one of the ninth 
and tenth features and is characterized in that when the 
coding information is transmitted, if there are a number of 
combinations of selectable coding information, the 
combination which minimizes the transmitted amount of 
information will be selected for transmission. 

Next, in a coding and decoding apparatus of the 
sixteenth aspect of the invention where the coding side 
transmits coded data together with identifying information 
for identifying the means of decoding the coded data, and 
the decoding side is capable of storing a plurality of 
decoding schemes so as to perform decoding based on one of 
the previously stored schemes, the coding and decoding 
apparatus is characterized in that: before transmitting the 
coded data to the decoding apparatus, the coding apparatus 
transmits the tools constituting an algorithm as the means 
of decoding the coded data, and the decoding apparatus 
reconstructs the algorithm using the tools so as to decode 
the received coded data based on the algorithm and stores 
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the tools therein; when the decoding apparatus receives the 
coded data which has been coded by the same tools, the 
decoding side decodes the coded data using the tools 
previously stored and the tools are defined in a 
hierarchical manner so that in place of a tool for a certain 
rank, the higher-ranked tool can be used to secure the 
minimum quality of the operation; and the coding apparatus 
on the transmitting side simultaneously transmits the 
decoding tool information and the coded data if the decoding 
apparatus on the receiving side has not decoding tool 
requested by the transmitting side. 

The seventeenth aspect of the invention resides in a 
coding and decoding apparatus having the sixteenth feature 
and is characterized in that when the decoding apparatus on 
the receiving side has no decoding tool requested by the 
coding apparatus on the transmitting side, the transmitting 
side temporarily changes the coding scheme using the coding 
tool that is in conformity with the decoding tool present 
on the receiving side. 

The eighteenth aspect of the invention resides in a 
coding and decoding apparatus having the sixteenth feature 
and is characterized in that when the decoding apparatus on 
the receiving side has no decoding tool requested by the 
coding apparatus on the transmitting side, the receiving 
side, whilst downloading the decoded tool transmitted from 
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the transmi-fting side to construct the requested decoding 
tool, temporarily decodes the coded data using a 
substitutable higher-ranked tool which is lowered in quality 
but still is able to perform decoding. 

The nineteenth aspect of the invention resides in a 
coding and decoding apparatus having the eighteenth feature 
and is characterized in that after the decoding tool 
requested has become prepared, the receiving side starts the 
decode operation using the requested decoding tool. 

According to the invention, the most suitable tools are 
selected based on the comparison between the 
tool-correspondent information stored in the 
tool-correspondent information storing means and the 
received tool -correspondent information. Thus, it becomes 
possible to perform the decoding operation efficiently and 
quickly. Particularly, an effective decoding operation is 
attained because the processing capacities for the tools are 
used as the tool-correspondent information and the tools, 
whose capacities fall within a permissible range for the 
decoding operation, are then selected. In this case, by 
transmitting the processing capacity which is numerically 
represented, it becomes possible to efficiently make a quick 
comparison in capacities. 

Further, it becomes possible to perform a quick 
selection of the tools because, by making the 
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tool -correspondent information be easy to compare, the tools 
of themselves do not need to be compared. Especially, the 
comparison and selection of the tools in the tool storage 
means can be performed quickly and effectively by allotting 
5 a unique key to each tool as the tool -correspondent 

information . 

Since the response control means is provided, only when 
there is a necessity for a toorl, it is possible to cause the 
coding apparatus on the other side to transmit the 

10 information of the tool. Thus, it is possible to shorten 

the time for transmission and reception and thereby perform 
the process efficiently. 

Further, since the coding and decoding tools are 
defined by hierarchization, and the data which conforms to 

15 the processing capacities of the thus hierarchized tools is 

attached with headers for identifying the processing 
capacity of the tool on the coding side^ the data can be 
reconstructed so as to have a data structure which conforms 
to the processing capacity of the decoding side and can be 

20 decoded. As a result, even if the processing capacities of 

the coding and decoding tools are not compatible, it is 
possible to decode the coded data. 

It becomes no more necessary for the decoding side to 
have all of various tools in order to deal with the data 

25 which is coded by such an algorithm having various coding 
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and decoding tools as a next -general: ion coding standard 
represented by MPEG4. Therefore, it is possible to reduce 
the cost of the apparatus. 

Further, since it is possible to alleviate the 
5 restriction on the hardware of the decoding apparatus, it 

becomes possible for a simple low-cost apparatus to decode 
the data coded based on the next-generation coding standard. 

In accordance with the scheme described above, the 
coding information, within the coding data stream 

10 transmitted from the coding apparatus, used for decoding the 

coded data is made up of only the identification code of the 
basic decoding scheme and the differential information with 
respect to the basic decoding scheme. Expanded decoding 
schemes from the basic decoding scheme can be represented 

15 by the differential information. That expansion of the 

basic decoding scheme is formed by designating the 
functional tools to be added to, canceled from or replaced 
in the basic decoding scheme. Further, if there are two or 
more methods in designating the differential information, 

20 the designating method which requires less amount of 

information is selected so that the amount of data to be 
transmitted becomes minimum. 

Further, in the coding and decoding apparatus described 
above, even if the receiving side has no tool having 

25 requested quality at the time of the first transmission, the 
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transmitting side starts coding and the receiving side 
starts decoding using other tools which are lower in quality 
as compared with the requested tools but still replaceable. 
The transmitting side simultaneously transmits the requested 
tools and the thus obtained coded data to the receiving 
side. The receiving side downloads the requested tools and 
prepares the tools whilst decoding the coded data using the 
other tools which are lower as compared with the requested 
tools in quality but still replaceable. When the 

establishment of the requested decoding tools is completed, 
the image communication with high quality requested will 
start. 



BRIEF DESCRIPTION OF THE DRAWINGS 

Fig.lA is a conceptual view showing the structure of 
a coding data stream which is formed by coding (compressing) 
image data based, on the H.261 schojne; 

Fig. IB is a conceptual view showing the structure of 
a coding data stream which is formed by coding (compressing) 
image data based on a coding scheme such as MPEG4 etc. whose 
algorithm is flexible; 

Fig.lC is a block diagram showing a conventional coding 
and decoding apparatus; 

Fig. 2 is a block diagram showing the configuration of 
coder based on H.261; 



- 38 - 



Fig. 3 is a block diagram showing the configuration of 
decoder based on H.261; 

Fig. 4 is a block diagram showing the structure of a 
coder which is provided with several kinds of algorithms; 
5 Fig ,5 is a block diagram showing the structure of a 

decoder which is realized by using a dedicated hardware 
device with software; 

Fig. 6 is a block diagram showing the structure of a 
decoder composed of a general -purpose processor and a 
10 compiler; 

Fig.7A is a diagram showing an arrangement of pixel 
data obtained by means of an inter- frame predictive coding 
tool of sampling per single pixel; 

Fig.7B is a diagram showing an arrangement of pixel 
15 data obtained by means of an inter- frame predictive coding 

tool of sampling per 1/2 pixel; 

Fig.7C is a diagram showing an arrangement of pixel 
data obtained by means of an inter-frame predictive coding 
tool of sampling per 1/4 pixel; 
20 Fig.SA is a diagram showing display images of a pattern 

TA obtained by decoding coded data based on the inter- frame 
predictive coding of sampling per single pixel; 

Fig .88 is a diagram showing display images of a pattern 
TA obtained by decoding coded data based on the inter- frame 
25 predictive coding of sampling per 1/2 pixel; 



Fig.SC is a diagram showing display images of a pattern 
TA obtained by decoding coded data based on the inter- frame 
predictive coding of sampling per 1/4 pixel; 

Fig -9 is a structural diagram showing a coding data 
stream which has coding information; 

Fig, 10 is a structural diagram showing a coding data 
stream which uses a decoding scheme identification code for 
coding information; 

Fig. 11 is a block diagram showing the first embodiment 
of a coding and decoding apparatus in accordance with the 
invention; 

Fig. 12 is an illustrative diagram showing the operation 
of the coding and decoding apparatus of the first 
embodiment ; 

Fig. 13 a diagram showing an example of a key 
assignment; 

Fig^.14 is a block diagram showing a coding apparatus 
in a coding and decoding apparatus of the second embodiment 
in accordance with the invention; 

Fig. 15 is a block diagram showing a decoding apparatus 
in a coding and decoding apparatus of the second embodiment 
in accordance with the invention; 

Fig. 16 is an illustration showing a coding apparatus 
and a decoding apparatus which are connected through a 
network; 
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Fig. 17 is a flowchart showing the steps of 
communication when a coding apparatus and a decoding 
apparatus are connected; 

Fig. 18 is a system block diagram showing a 
configuration of an overall transmitting and receiving 
system which is applied to the third embodiment of a coding 
and decoding apparatus of the invention; 

Fig. 19 is a conceptual view for explaining a 
hierarchical structure of a coded data obtained using an 
inter-frame predictive coding tool of sampling per 1/4 
pixel ; 

Fig. 20 is an operational flowchart for explaining the 
operation of inter-frame predictive coder; 

Fig.21A is a diagram showing the data structure of a 
bottom-ranked coded data which is added with headers in 
conformity with the rank of the inter- frame predictive 
coding tool used; 

Fig.21B is a diagram showing the data structure of 
Fig.21A in which headers are attached only to the leading 
part of pixel data; 

Fig. 22 is an operational flowchart for explaining the 
operation of an inter-frame predictive decoder; 

Fig. 23 is a diagram showing the structure of a coding 
data stream using a decoding scheme identification code and 
differential information in accordance with the forth 
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embodiment;; 

Fig .24 is a s1:ructural diagram showing an example of 
the coding apparatus of the fourt:h embodiment; 

Fig. 25 is a structural diagram showing an example of 
the decoding apparatus of the fourth embodiment; 

Fig. 26 is a structural diagram of a system showing the 
fifth embodiment of a coding and decoding apparatus of the 
invention; 

Fig ,27 is a flowchart for explaining the operation on 
the transmitting side in a coding and decoding apparatus of 
the fifth embodiment of the invention; and 

Fig. 28 is a flowchart for explaining the operation on 
the receiving side a coding and decoding apparatus of the 
fifth embodiment of the invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
_ The embodiments ^ of the present invention will 
hereinafter be described with reference to the accompanying 
drawings . 

( First embodiment ) 

Fig. 11 is a block diagram showing the first embodiment 
of a coding and decoding apparatus in accordance with the 
invention. This coding and decoding apparatus includes a 
controller 11a, a coder 12a, a response controller 13a, a 
decoder 14a, a tool storage controller 15a, a tool storage 
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section 16a, a coding capacity storing section 17a, a 
decoding capacity storing section 18a and a capacity 
comparator 19a. 

Controller 11a is a unit for controlling the entire 
apparatus. The total amount of the processing capacities 
for the tools in decoder. 14a which constitute the algorithm 
is stored previously in decoding capacity storing section 
18a as a decoding capacity. The processing capacity for 
each tool in decoder 14a is represented numerically. Coder 
12a transmits each tool together with its numerical 
processing capacity required for the tool to the other 
terminal device. 

Here, the capacity for a tool is represented 
numerically by selecting a standard operation step, 
determining the processing capacity for performing the 
standard operation step and comparing the processing load 
and capacity for the operation step of the ^tool with that 
of the standard operation step. 

The capacity for a tool is not numerically rated by 
coder 12a whenever the tool is transmitted but is designated 
previously by the implementer or by the selector based on 
its processing load. The numerical capacity is sent from 
controller 11a to the response controller where it is 
multi-processed with coded data and tool information so as 
to be transmitted. 
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Coding capacities received by decoder 14a are stored 
into coding capacity storing section 17a by individual tools 
which constitute the algorithm. The output from decoding 
capacity storage section 18a and the sum of the processing 
capacities for the tools constituting the algorithm are both 
inputted to capacity comparator 19a. 

Capacity comparator 19a compares the two values so as 
to Judge whether the operation of the algorithm using the 
tool received is permissible in the apparatus. The judgment 
can be made by means of a simple big-or-small comparator. 
If the operation can be determined as allowable from the 
comparison of the processing capacity, a permissible signal 
is sent out to tool storage controller 15a and the tool 
received is stored in tool storage section 16a so that it 
can be used for decoding the data which follows. 

If, from the comparison of the processing capacity, the 
decoding capacity of the apparatus is determined to be low 
to store the tool or the operation of the tool is determined 
as impermissible, the received tool is not useful so that 
a storage impermissible signal is sent out to tool storage 
controller 15a so as to prohibit the storage of the tool 
into tool storage section 16a. At the same time, 
instructions for transmitting the confirmation of whether 
the received tool can be downloaded for use are sent out to 
response controller 13a, 
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In general, in a coding scheme wherein the 
classification of the tools constituting its algorithm is 
defined, the decoding capacities on both sides are exchanged 
at the initial stage of the communication so that coding of 
data and transmission will be started after obtaining the 
decoding capacities of the other terminal devices. 
However, in a scheme wherein coded data is transmitted from 
the coding side together with tools constituting the 
algorithm for decoding the coded data, the load for the 
decoding process will remarkably vary depending upon the 
classification of the selected tools. Therefore, it is not 
considered that the capacity exchange of this kind is useful 
for previously obtaining the exact decoding capacity of the 
other terminal end. Therefore, in the latter scheme (the 
scheme of the invention), whenever a new tool is received, 
the capacity for the tool will be checked so as to determine 
whether the tool can be handled and the determined result 
will be transmitted to the opposite terminal. 

Fig. 12 is an illustrative diagram showing an 
operational example of the invention. Now, the operations 
of a coding apparatus 20a and a decoding apparatus 21a will 
be explained. Here, coding apparatus 20a and decoding 
apparatus 21a both are identical or composed of the coding 
and decoding apparatus shown in Fig. 11 and function as 
coding and decoding apparatuses, respectively. 
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First, it is assumed that coding apparatus 20a 
transmits a tool ' a * and its coding capacity ' Ca ' required 
for handling tool 'a' to the decoding apparatus. Decoding 
apparatus 21a stores the processing capacity Ca for the tool 
transmitted into coding capacity storage section 17a. The 
capacities for the tools constituting the algorithm are 
stored by individual tools. If the sura of the capacities 
other than Ca is assumed to Cz, the capacity required for 
the algorithm using the tool having capacity Ca can be 
represented by Ca + Cz. 

This value is compared with a decoding capacity ^ Cr ' 
which is previously determined in decoding apparatus 21a. 
In this comparison, if Ca + Cz < Cr, then tool 'a' which has 
been Just received is determined as permissible in the 
decoding side. If the tool can be used, it is downloaded 
and a signal indicating the completion of the download will 
be transmitted to the other side apparatus. Thus, coding 
apparatus 20a will be able to recognize that data using the 
tool transmitted therefrom can be transmitted. 

Next, it is assumed that coding apparatus 20a transmits 
a tool 'b' and its coding capacity 'Cb' required for 
handling tool 'b' to the decoding apparatus. Decoding 
apparatus 21a stores the processing capacity Cb for the tool 
transmitted into coding capacity storage section 17a. The 
capacities for the tools constituting the algorithm are 
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stored by individual tools. In the same manner as the above 
operation, the capacity required for the algorithm using the 
tool having capacity Cb can be represented by Cb + Cz . 

This value is compared with decoding capacity ' Cr ' 
which is previously determined. In this comparison, if Cb 
+ Cz > Cr, then the tool which has been just received is 
determined as impermissible in the decoding side- In this 
case, the download of the received will not be performed and 
a signal indicating the download error will be transmitted 
to the other side apparatus. Thus, the coding apparatus 
will be able to recognize that data using the transmitted 
tool cannot be transmitted. In response to this, the coding 
apparatus will send another tool in place, or will transmit 
data using tool 'a* which has been transmitted already so 
that the data can be decoded. 

As described above, the processing capacities for tools 

are used as the tool -correspondent information, and based 

oh the processing capacities, tools are selected so that the 

capacities for the selected tools fall within a permissible 

range for the decoding operation. Thus, it becomes possible 

to efficiently perform the decoding operation. In this 

■* • 
case, by transmitting the processing capacity which is 

numerically represented, it becomes possible to efficiently 

make a quick comparison in capacities. Further, since 

v/hether a received tool is permissible to use is judged 
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based on the capacity of the tool and only the tool which 
is usable will be downloaded^ it is possible to shorten the 
processing time required for the transmission and reception. 
( Second embodiment ) 

Table 2 shows an example of the control of tools in the 
second embodiment of a coding and decoding apparatus in 
accordance with the invention. As shown in Table 2, each 
tool is allotted with a unique key. For example^ a tool for 
motion compensation with integer-pixel precision is assigned 
with a key AaOl. In the same manner, all the tools are 
allotted with corresponding keys so that each tool can be 
discriminated . 

Fig. 13 shows an example of the key assignment. For 
instance, the key has a capital alphabetical character for 
broad classification (motion compensated prediction, DCT, 
etc.), a lowercase alphabetical character for subdivision 
( intager-pixel precision, half-pixel precision, etc.) and 
two digit numerals for version. 
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Table 2 



Key 


Tool 


AaOl 


Integer-pixel precision 
motion compensation 


AbOl 


Half -pixel precision 
motion compensation 


BaOl 


DCT 


BbOl 


Wavelet transform 


CaOl 


Linear quantization 


CbOl 


Vector quantization 



Fig. 14 is a block diagram showing a coding apparatus 
of the second embodiment in a coding and decoding apparatus 
in accordance with the invention. This coding apparatus 
includes: a controller 31 for controlling the entire 
apparatus; a tool storage section 33 for storing tools; a 
key table 32 for storing the memory locations in tool 
storage section 33 wherein the processing programs for the 
tools corresponding to different keys are stored; a 
processing unit 34 which reads out a processing program for 
a tool from the tool storage section and executes it; a work 
memory 35 for processing unit 34; and a network interface 
36 for performing communications with the outside. 

Fig. 15 is a block diagram showing a decoding apparatus 
of the second embodiment in the coding and decoding 
apparatus in accordance v/ith the invention. This decoding 
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apparatus includes: a controller 41 for controlling the 
entire apparatus; a key buffer 47 for temporarily storing 
a key received; a tool storage section 43 for storing tools; 
a key table 42 for storing the memory locations in tool 
storage section 43 wherein the processing programs for the 
tools corresponding to different keys are stored; a 
processing unit 44 which reads out a processing program for 
a tool from the tool storage section and executes it; a work 
memory 45 for processing unit 44; and a network interface 
46 for performing communications with the outside - 

Table 3 is an example of stored states of the tools in 
tool storage section 33 in the coding apparatus of Fig. 14 
as well as in tool storage section 43 in the decoding 
apparatus of Fig. 15. Table 4 shows an example of the 
correspondence of each key in key table 32 of Fig. 14 or in 
key table 42 of Fig. 15 to the memory location in tool 
storage section 33 or 43 for the tool corresponding to the 
key. 

The coding apparatus of Fig. 14 and the decoding 
apparatus of Fig. 15 can be connected through a network such 
as an ISDN, a mobile communication network, etc. by means 
of respective network interfaces 36 and 46, as shown in 
Fig. 16. 
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Table 3 



File No. 


Tool 


0 


Integer-pixel precision 
motion compensation 


1300 


Half -pixel precision 
motion compensation 


2500 


DCT 


4100 


Wavelet transform 


6000 


Linear quantization 


7700 


Vector quantization 



10 

Table 4 



Key 


Storage address 


AaOl 


0 


AbOl 


1300 


Bad 


2500 


BbOl 


4100 


CaOl 


6000 


CbOl 


7700 



Fig. 17 is a flowchart showing an example of the order 
of the communication steps when the coding^ apparatus of 
Fig. 14 and the decoding apparatus of Fig. 15 are connected 
25 in the manner shown in Fig. 16. First, an algorithm used in 

the coding apparatus as well as tools for constructing the 



algorithm is determined (Step SI). Then, the keys for the 
tools are transmitted to the decoding apparatus (Step S2). 
The decoding apparatus receives the keys and picks them up 
into key buffer 47 (Step Tl ) . The keys taken in to buffer 
47 are compared to key table 42 (Step T2 ) . If some of them 
are not registered, a request for tool transfer together 
with the keys is sent out to the coding apparatus (Step T3 ) . 
The coding apparatus transmits the processing programs for 
the tools corresponding to the keys (Step S4 ) . The decoding 
apparatus stores the processing programs for the tools in 
tool storage section 43 and registers the storage addresses 
with the keys into key table 42 (Step T5 ) . Then, the 
decoding apparatus transfers the tools from tool storage 
section 43 to processing unit 44 with reference to key table 
42 (Step T6). Next, the decoding apparatus sends out a 
request for coded data transfer to the coding apparatus 
(Step T7 ) . Receiving the request, the coding apparatus 
transmits coded data to the decoding apparatus (Steps S5 and 
S6 ) . The decoding apparatus receives the coded data (Step 
T8 ) and decodes it (Step T9 ) . When the coding apparatus has 
completed the transmission of all coded data, it sends out 
a signal indicating the end of transmission of coded data 
to the decoding apparatus (Step S7 and S8 ) . When the 
decoding apparatus receives the signal (Step TIO), the 
communication is ended. 
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In the above way, by using a unique key for each tool 
as an identifier for comparison, it become possible to 
select the required tools in a simple manner, thus making 
it possible to reduce the processing time. Further, keys 
are used to judge whether the same tools are registered in 
the decoding apparatus, and only when there is no 
corresponding tool, the tool is transferred. Consequently, 
it is possible to reduce the processing time for 
transmission and reception. 

As has been described, in accordance with this 
embodiment of the invention, the tool -correspondent 
information stored in the tool-correspondent information 
storing means is compared with the received 
tool-correspondent information so as to comparatively select 
the most suitable tools. Accordingly, it is possible to 
perform the decoding operation at a high efficiency in a 
short time. Particularly, the processing capacities for 
tools are used as the tool -correspondent information, and 
based on the processing capacities, tools are selected so 
that the capacities for the selected tools fall within a 
permissible range for the decoding operation. Thus, it 
becomes possible to efficiently perform the decoding 
operation. In this case, by transmitting the processing 
capacity which is numerically represented, it becomes 
possible to efficiently make a quick comparison in 
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capaci-ties . 

Further, since a simple and easily comparable 
identifier is used as the tool-correspondent information, 
it becomes possible to select required tools quickly without 
5 examining the content of tools. In particular, by allotting 

a key unique to each tool as the tool-correspondent 
information, it is possible to efficiently perform the 
process of comparatively selecting a tool from the tool 
storage means in a short time. 

10 Since the response control means is provided, only when 

there is a necessity for a tool, it is possible to cause the 
coding apparatus on the other side to transmit the 
information of the tool. Thus, it is possible to shorten 
the time for transmission and reception and thereby perform 

15 the process efficiently. 

( Third embodiment ) 

In reference to drawings, description will be made of 
a coding and decoding apparatus of the third embodiment of 
the invention hereinbelow in which a video signal is sampled 

20 per 1/4 pixel by the inter- frame predictive coding so that 

the coded data is transmitted and the transmitted data is 
decoded . 

Fig. 18 is a system block diagram showing the 
configuration of an overall transmitting and receiving 
25 system to which a coding and decoding apparatus of the 
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invention is applied. A transmitting system la shown in the 
figure includes: a video input unit 101 converting video 
information inputted through unillustrated visual sensor 
into a video signal; a coding apparatus 102 coding the 
inputted video signal from video input unit 101 to produce 
coded data; a transmitter 103 outputting the coded data 
produced by coding apparatus 102 onto a communication net 
300. 

Here, coding apparatus 102 is composed of: a 
transformer 121 receiving the video signal and subjecting 
it to the DCT (Discrete Cosine Transform); a quantizer 122 
quantizing the DCT coefficients obtained by the DCT in 
transformer 121; an inter-frame predictive coder 123 
receiving the video data quantized through quantizer 122 and 
subjecting it to an inter-frame predictive coding process. 
Inter- frame predictive coder 123 has an inter- frame 
predictive coding tool EC^ which samples the data per single 
pixel, an inter-frame predictive coding tool EC2 which 
samples the data per half pixel, and an inter- frame 
predictive coding tool EC^ which samples the data per 
quarter pixel, so that one of these inter- frame predictive 
coding tools will be selected in accordance with the quality 
of display image required on the decoding side. 

In the case of transmitting system la shown in Fig. 18, 
inter-frame predictive coding tool EC^ of sampling per 1/4 
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pixel is selected in inter-frame predictive coder 123, That 
is, the coding tool which provides the highest quality of 
display image is selected. 

A receiving system 2a is composed of: a receiver 201 
5 serving as the input interface for picking up the coded data 

of image on communication net 300 which has been transmitted 
from transmitting system la; a decoding apparatus 202 
receiving the coded data from receiver 201 and decoding it 
into a video signal; and a video output unit 203 receiving 

10 the video signal from decoding apparatus 202 and displaying 

the video information on unillustrated monitor. Here, 
decoding apparatus 202 is composed of: an inter- frame 
predictive decoder 221 subjecting the coded data to an 
inter- frame predictive decoding process to produce image 

15 data; inverse quantizer 222 inverse quantizing the image 

data obtained by decoding from inter- frame predictive 
decoder 221 and getting DCT coefficients; and an inverse 
transformer 223 transforming the DCT coefficients produced 
by inverse quantizer 222 into a video signal. In the case 

20 of receiving system 2a shown in Fig. 18, inter- frame 

predictive decoder 221 has only an inter- frame predictive 
decoding tool DC^ which samples the data per half pixel. 

The operations of coding apparatus 102 and decoding 
apparatus 202 will be described hereinbelow by explaining 

25 the operations of transmitting system la and receiving 
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system 2a t:hus configurated. 

First, the operation of transmitting system la will be 
described. Video input unit 101 converts video information 
inputted through a visual sensor such as a camera etc. into 
5 a video signal and supplies it to coding apparatus 102. The 

video signal inputted to coding apparatus 102 is converted 
into image data through transformer 121 and quantizer 122. 
Inter- frame predictive coder 123 receives the image data and 
subjects it to an inter- frame predictive coding process to 

10 produce coded data. This coded data is supplied to 

transmitter 103. Transmitter 103 outputs the coded data 
onto communication net 300. 

Here, inter-frame predictive coder 123 produces coded 
data in a hierarchical structure in accordance with the 

15 coding capacity of the inter- frame predictive coding tool 

used for coding. Before the detailed description of the 
operation of this inter-frame, predictive coder 123, the 
hierarchization of coded data will be explained. In 
hierarchizing coded data, the ranks of the inter- frame 

20 predictive coding tools used for producing coded data are 

to be defined. That is, when the coded data which is 
obtained by the first inter-frame predictive coding tool for 
a certain rank contains the coded data which is obtained by 
the second inter- frame predictive coding tool for another 

25 rank, the rank of the former inter-frame predictive coding 
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tool is defined -to be lower than the rank of the latter 
inter- frame predictive coding tool. According to this 
definition, the inter-frame predictive tools of sampling per 
single pixel, per 1/2 pixel and per 1/4 pixel belong to the 
5 "top, medium and bottom ranks, respectively, based on the 

inclusionary relation of the coded data. 

Fig. 19 is a conceptual view for explaining the 
hierarchical structure of the coded data obtained using the 
inter-frame predictive coding tool of sampling per 1/4 

10 pixel. In the figure, top-rank coded data D^^, 

medium-ranked coded data D - and bottom-ranked coded data D ^ 
are produced by the aforementioned top-ranked, medium-ranked 
and bottom-ranked inter- frame predictive coding tools, 
respectively. Pixel data a^ to d^, to i^ and to y^ 

15 shown in Fig.7C correspond to top-ranked coded data D^. , 

medium-ranked coded data D^^ bottom-ranked coded data 

D^^, respectively. As understood from Fig. 19 and Figs.7A to 
7C, the bottom-rank coded data has such a data structure 
that it hierarchically includes the medium and top-ranked 

20 coded data. The medium-ranked coded data has such a data 

structure that it hierarchically includes the top-ranked 
coded data. 

The hierarchization of coded data by the inter- frame 
predictive coding process is obtained by noticing the 
25 hierarchical structure of the coded data as stated above. 
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That is, coding apparatus 102 selectively adds a header 
(identifiers) for identifying the rank of data, to each 
piece of pixel data a^ to y^ constituting coded data D^^ of 
the bottom rank, so that the coded data whose rank is 
defined in correspondence with the coding capacity of the 
inter-frame predictive coding tool used is created. 

Now, the operation of inter-frame predictive coder 123 
will be described in detail with reference to the flowchart 
shown in Fig. 20. First, inter-frame predictive coder 123 
inputs image data from quantizer 122 (Step SOI), and then 
it selects an image- frame predictive coding tool in 
accordance with the quality of display image on the decoding 
side (Step S02). Since, in inter-frame predictive coder 
123, inter-frame predictive tool EC^ of sampling per 1/4 
pixel (the inter-frame predictive tool for the bot:tom rank) 
which provides high-quality image display has been selected 
as already mentioned, the image data is subjected to the 
inter- frame predictive coding process by means of 
inter-frame predictive coding tool EC^ (Step SOS). Each 
piece of image data a^ to y^ thus obtained is added with one 
of headers HD^^ to HD^ which corresponds to the rank of the 
inter- frame predictive coding tool (Step S04 ) . Thus, coded 
data D^^ of the bottom rank is produced. Then, as the coded 
data is outputted to transmitter 103, the operation is 
ended . 
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In the above Step S02, when the high-quality display 
image is not requested, the inter- frame predictive coding 
process is performed by using the inter-frame predictive 
coding tool of sampling per 1/2 pixel or single pixel (Step 
SOS or S07). Thereafter, the thus obtained coded data is 
added with corresponding headers (Step S06 or SOS). 

Now, headers HD^ to HD^ will be explained. Figs.21A 
and 2 IB are diagrams showing the data structures of 
bottom-ranked coded data D^^ where each piece of coded data 
is added with any of headers HD^ to HD^ which corresponds to 
the rank of the inter-frame predictive coding tool. More 
specifically, there figures show the data structures 
wherein, from pixel data a^ to y^ shown in Fig.7C, each 
piece of pixel data a^ to arranged on the first and 

second lines is added with one of headers HD, to HD, . In 
Fig.21A, header HD^^ is attached to the pixel data (the pixel 
data of the pixels indicated by ' + ' in Fig.7C) corresponding 
to the data which is produced by the inter- frame predictive 
coding of sampling per single pixel. Next, header is 
attached to the pixel data ( the pixel data of the pixels 
indicated by 'o' in Fig.7C) corresponding to all the data 
which is produced by the inter-frame predictive coding of 
sampling per 1/2 pixel, except the pixel data produced by 
the inter- frame predictive coding of sampling per single 
pixel. Further, header HD^ is attached to the pixel data 
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(the pixel data of the pixels indicated by 'a' in Fig.7C) 
corresponding to all the data which is produced by the 
inter-frame predictive coding of sampling per 1/4 pixel, 
except the pixel data produced by the inter-frame predictive 
coding of sampling per 1/2 pixel. 

In this way, when each piece of pixel data constituting 
coded data D^^ is attached with the header corresponding to 
the rank of the inter-frame predictive coding tool, the set 
of pixel data added with header HD^ becomes equal to the 
content of the coded data which is obtained by inter- frame 
predictive coding tool EC^ of the top rank. The set of 
pixel data added with header HD^ or becomes equal to the 
content of the coded data which is obtained by inter- frame 
predictive coding tool of the medium rank. Therefore, 

on the decoding side, pixel data a^^ to d^, to i^ and a^ to 
y^ which correspond to respective inter- frame predictive 
goding tools of different ranks, among coded data D^^ can be 
discriminated by identifying the class of headers attached 
to pixel data a^ to y^ constituting coded data D^^. Further, 
it is possible to identify the coding tools used for 
producing coded D^^ by examining the classes of the headers. 
In this way, coded data D^^ of the bottom rank which is 
constituted by the pixel data stream with headers attached 
is sent out onto communication net 300 by means of 
transmitter 103. 
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As shown in Fig.21B, when a series of pixel data is 
added with the same headers, the headers may be attached to 
only the leading part of the pixel data. This attachment 
of headers will inhibit the increase of the amount of data 
5 due to the addition of headers. 

Next, the operation of receiving system 2a will be 
described. Receiver 201 as the input interface of receiving 
system 2a picks up coded data D^^ from communication net 300 
and supplies it to decoding apparatus 202. Coded data D^^ 

10 taken in to decoding apparatus 202 is decoded in inter- frame 

predictive decoder 221 to be converted into image data. The 
thus obtained image data after the conversion is further 
converted into a video signal through inverse quantizer 222 
and inverse transformer 223. Video output unit 203 develops 

15 the video signal into a visual image as video information 

on the monitor. Here, inter- frame predictive decoder 221 
has only inter- frame predictive decoding tool DC^ of 
sampling per 1/2 pixel which has the decoding capacity 
corresponding to coding capacity of inter- frame predictive 

20 coding tool EC2. This inter-frame predictive decoding tool 

DC2 is defined as a tool of the medium rank. 

The operation of inter-frame predictive decoder 221 
will hereinbelow be described in detail with reference to 
the operational flowchart shown in Fig. 22. Inter- frame 

25 predictive decoder 221 receives coded data D from receiver 
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201 (Step Sll). Then, it is judged whether the rank (coding 
capacity) of the coding tool of the inter- frame predictive 
coding tool used in coding is equal to the rank (decoding 
capacity) of inter- frame predictive decoding tool 
5 equipped in inter-frame predictive decoder 221 (Step S12). 

This judgment can be made by identifying header HD^ among 
headers HDl to HD4 attached to the coded data. Since input 
coded data D^^ is of the bottom rank, the rank of the input 
coded data is determined not to be identical with the rank 

10 (medium rank) of decoding tool DC^ equipped in inter- frame 

predictive decoder 221 (Step S12, No). 

Next, it is judged whether the rank of coding tool EC^ 
(bottom-ranked) which has been used to produce coded data 
is lower than the rank of decoding tool 

15 (medium-ranked) equipped in inter- frame predictive decoder 

221 (Step S15). In this judgment, if it is on the lower 
side (Step S15, Yes), only the pixel data which has been 
coded with headers HD^^ and which conform with the 

decoding capacity of inter- frame predictive decoding tool 

20 DC2 of the medium rank, is extracted from coded data D^^ to 

reconstruct coded data (Step S16). 

Specifically, from pixel data a^ to y^ obtained by the 
inter-frame predictive coding process of sampling per 1/4 
pixel, the pixel data which corresponds to the coded data 

25 obtained by the inter-frame predictive coding of sampling 



- 63 - 



per 1/2 pixel is picked up to reconstruct coded data (Step 
S16)- The extraction of data is performed by identifying 
the header added to each piece of pixel data so that the 
pixel data added with header HD^ or HD^ is extracted from 
pixel data a^ to y^. Then, the headers are removed from the 
thus extracted data, whereby coded data D^^ is reconstructed 
into coded data D^^- Coded data D^^ thus reconstructed is 
made to undergo the inter- frame predictive decoding by using 
tool DC2 of the medium rank (Step S17 ) , 

Suppose that inter- frame predictive decoder 221 has 
unillustrated inter-frame predictive decoding tool DC^ of 
sampling per 1/4 pixel. In this case, it is determined in 
the judgment at Step S12 that the rank of decoding tool 
equipped on the decoding side is the same as that of the 
coding tool (Step 812, Yes), and headers HD^ to HD^ are 
removed from coded data D^^, whereby coded data D^^ without 
headers is reconstructed (Step S13). Then, inter-frame 
predictive decoding tool DC^ is used to cause reconstructed 
coded data D^^ to undergo to the inter- frame predictive 
decoding process. The image data obtained from this 
decoding process is outputted to inverse quantizer 222 (Step 
S14), and thus the decoding operation is ended. 

Further, if the inter-frame predictive coding is 
executed using inter-frame predictive coding tool EC, of 
sampling per single pixel, the received coded data must be 
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coded data D^^ which belongs to the higher rank. Therefore, 
in the above judgment at Step S15, the rank of the coding 
tool is determined not to be on the lower side (Step S15, 
No). In this case^ since coded data D^^ which conforms with 
inter- frame predictive decoding tool equipped in the 

decoding side can not be reconstructed from coded data D^^ 
which is received, the operation is ended without executing 
the decoding process . 

As has been apparent from the foregoing description, 
in accordance with the coding and decoding apparatus of the 
third embodiment, n-ranked coded data which is produced 
using an n-ranked coding tool and decoded using an n-ranked 
decoding tool has a hierarchical structure which includes 
(n+1) -ranked coded data which is produced using a 
(n+1) -ranked coding tool and decoded using a (n+1) -ranked 
decoding tool. This inclusionary relation holds for an 
arbitrary positive integer n. 

On the coding side which has the n-ranked coding tool, 
the coding means produces the n-ranked coded data using the 
n-ranked coding tool. The identifier adding means attaches 
N-ranked identifiers to N-ranked coded data which is 
included in . the n-ranked coded data but is other than 
(N+1) -ranked coded data included in the N-ranked coded data. 
In this case, N is a positive integer which is equal to n 
or more and whose upper boundary is limited depending upon 
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the depth of the hierarchy of the coded data. As a result, 
if n = 1, for example, then first-ranked identifiers are 
attached to first-ranked coded data except for second-ranked 
coded data included in the first-ranked coded data. The 
5 second-ranked identifiers are attached to the second-ranked 

coded data included in the first-ranked coded data except 
for the third-ranked coded data included in the 
second-ranked coded data. Similarly, the fourth or 
fifth-ranked identifiers are attached to the data in 

10 accordance with the depth of the hierarchy of the coded data 

included in the first-ranked coded data. 

Next, on the decoding side which has a m-ranked 
decoding tool (m is a positive integer satisfying m > n), 
the data reconstructing means extracts a set of the N-ranked 

15 coded data which is included in the n-ranked coded data and 

attached with the N-ranked identifiers where N > m. By this 
extraction, if n = 1, m = 2, for example, second-ranked 
coded data included in first-ranked coded data (a set of the 
coded data with second, third-ranked, identifiers 

20 attached) will be extracted. If n = 1, m = 3, as another 

example, third-ranked coded data included in the first coded 
data (a set of the coded data with third, fourth-ranked, • • • • 
identifiers attached) will be extracted. From the thus 
extracted set of the N-ranked coded data with N-ranked 

25 identifiers attached, the m-ranked coded data is 
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reconstructed. Then, the decoding means decodes the 
m-ranked coded data using the m-ranked decoding tool. Here, 
it is possible to perform the coding and decoding operations 
using an inter- frame predictive coding tool for the coding 
tool and an inter- frame predictive decoding tool for the 
decoding tool . 

As stated above, since the coding and decoding tools 
are defined by hierarchization, and the data which conforms 
to the processing capacities of the thus hierarchized tools 
is attached with headers for identifying the processing 
capacity of the tool on the coding side, the data can be 
reconstructed so as to have a data structure which conforms 
to the processing capacity of the decoding side and can be 
decoded. As a result, even if the processing capacities of 
the coding and decoding tools are not compatible, it is 
possible to decode the coded data. 

It becomes no more necessary for the decoding side to 

have all of various tools in order to deal with the data 

which is coded by such an algorithm having various coding 

and decoding tools as a next-generation coding standard 

represented by MPEG4. Therefore, it is possible to reduce 
« 

the cost of the apparatus . 

Further, since it is possible to alleviate the 
restriction on the hardware of the decoding apparatus, it 
becomes possible for a simple low-cost apparatus to decode 
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the data coded based on the next-generation coding standard. 
(Fourth embodiment) 

The fourth embodiment of the invention will be 
described hereinbelow with reference to drawings - 

Fig. 23 is a diagram showing an example of the structure 
of a coding data stream used in the coding and decoding 
apparatus of the invention. 

In Fig. 23^ a decoding scheme identification code 431 
as a component of the coding information is the same as that 
of Fig. 10, and is an identification code for a previously 
defined basic decoding scheme. Additional condition 
information 432 and tool information 433 are instructional 
information for adding, deleting or replacing certain 
procedures for the basic decoding scheme identification code 
431. That is, when, for example, the coding scheme which, 
although it basically conforms to H.261, is partially 
modified at minute points is used, only the decoding scheme 
identification code of H.261 with the differential 
information from the scheme is transmitted as the coding 
information in order to enable the reconstruction of the 
decoding scheme on the decoding side. In this way, it is 
possible to eliminate the loss of transmitting all the 
functional tool information as shown in Fig. 9. 

As stated above, the additional condition information 
may include specifying conditions such as addition, deletion 
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and replacement;, etc. For example, the transaction of 
motion compensation used in the H.261 coding scheme to 
efficiently display the movement of objects, tends to lower 
the speed of processing since a lot of operations are needed 
on the coding side. In order to avoid such a situation, it 
is possible not to use the motion compensation. In such a 
case, the additional condition information can be provided 
with an instruction of deleting a functional tool while the 
functional tool information can be provided with the 
identification code of the motion compensating tool. In 
this way, it is possible to drastically reduce the 
transmitted amount of data compared to the case where the 
revised coding information based on H.261 which does not 
have the motion compensation is transmitted from the 
beginning . 

Similarly, other instructions such as addition of a 
functional tool which is not included in the basic decoding 
scheme or replacement of a functional tool with another 
functional tool which has a similar function, can be made 
by the combination of the additional condition information 
and the functional tool information. Thus, it is possible 
to reduce the transmitted amount of data compared to the 
case where all the information which constitutes the 
decoding scheme is transmitted from the beginning. 

Now, the configurations of the coding apparatus and the 
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decoding apparatus for prac-ticing -the above operations will 
be described hereinbelow. 

Fig, 24 is a block diagram showing an example of the 
embodiment of the coding apparatus having the function 
stated above. In Fig. 24, a reference numeral 441 designates 
a coding scheme database , which has basic coding schemes 
and a group of functional tools stored therein so that they 
can be read out from the outside. Designated at 442 is a 
coding scheme selector which selects the most preferable 
coding scheme in accordance with the nature of the input 
data, and reads out the components of a required coding 
scheme and the content of functional tools. The thus 

obtained coding scheme is transferred to a coding section 
443 where the input data is sequentially coded. The control 
of the processing tools for the aforementioned sections as 
well as data input/output control etc., is performed 
altogether in a coding controller 444. If, in coding scheme 
selector 442, there are a number of selecting ways of the 
coding information representing the determined scheme, the 
selector compares in the transmitted amount of data between 
different selecting ways and selects the one whose 
transmitted amount of data is minimized and transmits it, 
in order to avoid the reduction in the efficiency in the use 
of the communication line as much as possible. 

Fig. 25 is a block diagram showing an example of the 
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embodiment: of the decoding apparatus having the function 
described above. In Fig. 25, a reference numeral 451 
designates a decoding scheme database , which has basic 
coding schemes and a group of functional tools stored 
therein so that they can be read out from the outside. 
Designated at 452 is a decoding scheme interpreter which 
reads out the components of a required decoding scheme and 
the content of functional tools from decoding scheme 
database 451. The thus decoding scheme is transferred to 
a decoding section 453 where the input coded data is 
sequentially decoded. The control of the processing tools 
for the aforementioned sections as well as data input/output 
control etc. , is performed altogether in a decoding 
controller 454. 

As has been apparent from the foregoing description, 
the fourth embodiment of the invention offers the following 
advantages . 

First, since when the coding information is 
transmitted, an identification code of a previously defined 
basic decoding scheme and the differential information from 
the basic decoding scheme are transmitted, it is possible 
to reduce the transmitted amount of data compared to the 
case where the information of the functional tools for the 
decoding scheme and other information are all determined, 
and the efficiency in the use of the communication line will 
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be preven-fced from being deteriorated very much. 

Next, since when the coding information is transmitted, 
the information that one or some kinds of functional tools 
will be added to the basic decoding scheme, is transmitted 
5 as the differential information, the decoding scheme 

incorporated in the decoding apparatus can be expanded for 
use. 

Since when the coding information is transmitted, the 
information that one or some kinds of functional tools will 

10 not be used, is transmitted as the differential information, 

the decoding scheme incorporated in the decoding apparatus 
can be simplified for use. 

Further, since when the coding information is 
transmitted, the information that one or some kinds of 

15 functional tools will be replaced with another or others, 

is transmitted as the differential information, the decoding 
scheme incorporated in the decoding apparatus can be 
modified for use. 

Finally, when the coding information is transmitted, 

20 if there are a number of combinations of selectable coding 

information, the amount of combined information is compared 
with the others so that the combination which minimizes the 
transmitted amount of information can be selected for 
transmission. Accordingly, it is possible to inhibit the 

25 efficiency in the use of the communication line from being 



degraded very much. 
(Fifth embodiraeni: ) 

The fifth embodiment of the invention will be detailed 
with reference with drawings . For the description 

hereinbelow, the inter- frame predictive coding and decoding 
method will explained as the illustrative configuration of 
a hierarchical coding and decoding tool . In the 

description, the method of sampling per single pixel, the 
method of sampling per 1/2 pixel and the method of sampling 
per 1/4 pixel are assumed to be in the descending order from 
the top-rank. 

Fig. 26 is a system block diagram showing an example of 
the fifth embodiment of the invention wherein lb and 2b 
designate coding and decoding apparatuses and 603 a 
communication net. Coding and decoding apparatus lb 
includes a video input unit 611, a switch 612, a transformer 
613, a quantizer 614, an inter-frame predicting coder 615, 
a transmitter 616, a tool information transmitter/receiver 
617 and a tool information/downloading controller 618. 
Coding and decoding apparatus 2b includes a video output 
unit 621, a switch 622, an inverse transformer 623, an 
inverse quantizer 624, an inter-frame predicting decoder 
625, a receiver 626, a tool information transmitter/receiver 
627 and a tool information/ downloading controller 628. 

Now, description will be made of a case where an 
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input;t:ed video image is t:ransmitt:ed through communication 
net 603 from coding and decoding apparatus lb to coding and 
decoding apparatus 2b and the received signal is outputted. 
Fig. 27 is a flowchart of the operation in coding and 
decoding apparatus lb. Fig. 28 is a flowchart of the 
operation in coding and decoding apparatus 2b. 

When a video image is inputted to video input unit 611 
in coding and decoding apparatus lb ( Step S21 ) , the video 
signal is coded through transformer 613, quantizer 614 and 
inter-frame predictive coder 615, each having a number of 
tools. Each of the tools are previously hierarchized in 
such a manner that the tools which are replaceable with the 
other tools are defined as bottom-ranked tools and the tools 
which cannot be replaced with the other tools are defined 
as top-ranked tools. In this embodiment, as shown in 
Fig. 26, the tools from the top-rank to the bottom-rank, are 
designated with A, B and C, for convenience. In this 
example, as shown in Fig. 26, the inter- frame predictive 
coding tools are arranged so that the inter-frame predictive 
coding tool of sampling per single pixel (tool A), the 
inter- frame predictive coding tool of sampling per 1/2 pixel 
(tool B) and the inter- frame predictive coding tool of 
sampling per 1/4 pixel (tool C) are from the top to the 
bottom. Therefore, each of the tools in transformer 613, 
quantizer 614 and inter-frame predicting coder 615 are 
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selec-ted through switch 612 in conformity with the quality 
of the coded video data to be transmitted from coding and 
decoding apparatus lb. 

In this embodiment, the inter- frame predictive coding 
5 of sampling per 1/4 pixel is categorized as the highest 

quality while the inter-frarae predictive coding of sampling 
per single pixel is categorized as the lowest quality. 

In accordance with this example, first, tool C from 
each section is selected as the highest quality tool by 

10 switch 612 (Steps S22, S23 and S24). The inputted video 

image is processed through selected tools C, in particular, 
the signal is made to undergo the inter-frame coding tool 
of the tool of sampling per 1/4 pixel. The thus processed 
signal is transmitted onto communication net 603 (Step S25). 

15 This coding information on communication net 603 is accepted 

by means of coding and decoding apparatus 2b. At that 
moment, coding and decoding apparatus lb acquires the 
knowledge of the state of coding and decoding apparatus 2b 
from tool information transmitter/receptor 617 on the 

20 transmitting side and tool information transmitter/receiver 

627 on the receiving side, and constructs the required tool 
information in tool information/downloading controller 618 
(Step S26). The thus obtained tool information is sent out 
from tool information transmitter/receptor 617 v/hilst the 

25 coding information is transmitted from the transmitter 616 



(Step S27 ) . 

At this point, until the tools in coding and decoding 
apparatus 2b become ready (Step S28), coding and decoding 
apparatus lb may temporarily change its tools (Step S29 ) to 
transmit coded data (Step S30), Thus, since the 

transmitting side, in place of performing a heavy process 
using the tools for high quality requested, may perform a 
light-loaded process using the tools for low quality, and 
since the receiving side need not take the trouble to modify 
the coded data of high quality into coded data of low 
quality, these facts will relieve the burdens on both the 
apparatuses. In this example, this temporal coding is 
performed using the inter- frame predictive coding tool of 
sampling per single pixel (Step S24). 

In this embodiment, in the coding and decoding 
apparatus 2b on the decoder, inverse converter 623 and 
inverse quantizer 624 have their respective tools C as the 
decoding tools requested, but inter- frame predictive decoder 
625 dose not have the requested tool of sampling per 1/4 
pixel. For this reason, as shown in the flowchart in 
Fig. 27, coding and decoding apparatus lb on the transmitting 
side transmits the tool information whilst sending out 
either the coding information of low quality which is 
prepared by the modified tools or the coding information of 
the original quality. As shown in Fig. 28, coding and 
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decoding apparatus 2b on the decoder receives coded data 
(Step S31). Since no requested tool is found in the decoder 
(Step S32), the decoding is performed by using the 
top-ranked tools (Step S33 ) although the quality becomes 
degraded from the requested level. At the same time, the 
apparatus 2b receives the tool information of the requested 
tool by tool information transmitter/receiver 627 (Step S34) 
so as to construct the tool in the tool information/ 
downloading controller 628 (Step S35 to S37). 

In this way, as soon as the requested tool becomes 
ready, the operation is switched from the top-ranked tool 
to the requested tool by means of switch 622 so that the 
coded data of the requested quality can be decoded thus 
displaying the video output by means of video output unit 
621. In this example, whilst the coded data which has been 
coded by the inter-frame predictive coding of sampling per 
1/4 pixel is decoded temporarily by means of inter-frame 
predictive coding of sampling per single pixel, the tool for 
the inter-frame predictive coding of sampling per 1/4 pixel 
is downloaded, and as soon as the tool becomes prepared, the 
coded data will be decoded by the inter- frame predictive 
coding of sampling per 1/4 pixel. 

As has been described above, in accordance with the 
fifth embodiment of the invention, when the coded data is 
transmitted, if the tools which barely permit the decoding 
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of the data although the decoded data degrades, is found in 
the receiving side, the transmitting side transmits the 
requested tools constructing the algorithm as the means of 
decoding the coded data together with the coded data. The 
transmitting side temporality transmits the coded data which 
is made to conform to the tools barely permitting the 
decoding though the decoded data degrades, whilst sending 
out the tool information. Further, the receiving side, 
whilst decoding using the tools which barely permit the 
decoding although the data degrades, downloads the requested 
tools. Moreover, after the requested tools become prepared, 
the receiving side decodes the coded data of the requested 
quality. 

In conclusion, as in the next-generation image coding 
scheme (MPEG4 etc.), when a variety of coding and decoding 
tools are used in the algorithm, the coding and decoding 
tools which^ can be replaced by other tools although the 
quality degrades if the substitution is used, are defined 
as the bottom-ranked tools for the highest quality and the 
tools which are not replaceable are defined as the 
top-ranked tools. In this way, the apparatus is 

hierarchized so that the interchangeability is established 
based on the top-ranked tools and the tool information can 
be downloaded. In such a coding and decoding apparatus of 
the fifth embodiment of the invention, if the decoding 
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apparat:us on the receiving side has no tool requested by the 
transmitting side, the coding apparatus on the transmitting 
side will transmit decoding tool information and coded data 
simultaneously. Therefore, compared to the configuration 
in which only the tool information is transmitted at first, 
and the coded data is decoded based on the tool only after 
the tool information is downloaded, it becomes possible to 
save the time during which the tool information alone is 
transmitted. As a result, it is possible to quicken the 
start of transmission of coded data. 

Since when the decoding apparatus on the receiving side 
has no decoding tool requested by the transmitting side, the 
coding apparatus on the transmitting side temporarily 
changes the coding scheme using the coding tool that is in 
conformity with the decoding tool present on the receiving 
side, it is possible to reduce the delay of the start of 
decoding although the quality temporarily degrades. 

When the decoding apparatus on the receiving side has 
no decoding tool requested by the coding apparatus on the 
transmitting side, the receiving side, whilst downloading 
the decoding tool transmitted from the transmitting side to 
construct the requested decoding tool, temporarily decodes 
the coded data using a substitutable top-ranked tool which 
is lowered in quality but still is able to perform decoding. 
Therefore, the top-ranked tool can be used at the early 
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stage thus making it possible to reduce the delay of the 
start of decoding although the quality temporarily degrades. 
Moreover, since after the decoding tool requested has 
become prepared, the receiving side starts to decode using 
the requested decoding tool, the quality can be assured 
after the completion of the construction of the requested 
decoding tool . 



